In a business decision-making environment, the importance and the need to collect metrics to measure business performance is well known and obvious. Metrics are typically computed using raw data collected from multiple source systems, from activities that are usually associated with sales, marketing, finance, supply chain, service efficiency, manufacturing, production, quality, customer care and so on. These metrics are presented in the form of graphs, reports, analytical results, dashboards etc., to senior executives. Senior executives make use of these artifacts for their business reviews, (sometimes referred to as metrics Reviews or operations reviews) so that they can take timely decisions based on metrics to run their business.
Regardless of the type of business and the type of metrics, the process of raw data collection, aggregation, analysis, presentation, and tracking actual performance metrics against target metrics—they all involve the following five stages: a) Data collection, data aggregation and cleansing, b) Storing raw data and metrics in a data warehouse or metric data mart, c) Organization of metrics, according to business impacts and rules, d) Using Business Intelligence (BI) tools to create graphs and reports and dashboards, and e) Collaborating among peer managers and leaders and discussing business impacts, raising issues, throwing action items, tracking milestones to achieve target metrics etc.
Today, the Industry makes use of five distinct types of tools to achieve every stage above. Different variations of the five types of tools may exist, whereby some steps are either skipped, or performed by the same tool, or some stages are done manually to fill the process gaps. It is important to note that these five steps are accomplished largely with the help of programming using developers, not the end business user. The above five stages are explained in detail below.
Stage 1: Data aggregation and cleansing: also known as extract, transform, and load (ETL), currently accomplished only by ETL development tools run by professional programmers, not directly by end users or novices without programming knowledge.
Stage 2: Data warehousing or data mart: every enterprise builds its own customized data warehouse or logical data structure specific to their organizational needs using well-known relational database management systems (RDBMS).
Stage 3: Organizing a cascaded set of metrics: defining the relevant set of metrics, and cascading them from Key Performance Indicators (KPI metrics), down to detailed tactical levels of metrics measurements. Metrics are also mapped to business drivers, risks, business functions and business priorities, as well as attributes like organization, product service, geographic regions etc. Every metric is mapped to respective metric owners who are responsible for delivering to that metric, as well as metric consumers (or users) authorized to access the metric. In addition, should a metric fall outside the allowed band or Red/Yellow/Green limits defined, appropriate notification and alerts such as emails and escalation chains get sent out.
Stage 4: Use of Business Intelligence (BI) tools for the construction of graphs such as time-based trend charts, pie charts showing metrics sliced by attributes (such as revenue per business unit, sales by region, sales by product etc.,). End users may desire a variety of visualizers such as stacked column charts (graphical representation of pivot tables) or two-dimensional grids or three-dimensional cubes for their decision-making purposes. Furthermore, dashboards comprising one or more such charts are constructed along with necessary data grids. The dashboards in turn offer the ability to drill down from top levels to lowest level transactions that led to the metric. Such drill downs generally fall into at least one or more of three ways: time-based drill down (for example, drill down from quarterly sales to monthly breakdown and then drill down to sales by week) or attribute-based drill down (for example, Europe revenue broken down by countries within Europe, or Europe total revenue broken down by products) or component-based drill down, such as from a total revenue into its components (total Revenue=gross revenue plus carry over payments, minus adjustments, minus tax etc.). In current market, the above tasks of construction of charts, compilation of charts into dashboards, and the ability to drill down to details are typically done by developers. It is important to note that many variations of these drill-down capabilities are created by developers specific to their organization. There exists no configurable, off-the-shelf function that can be done on the fly, ready to be used by end users, with automatic, intuitive drill-down capabilities. Stage 5: Collaboration: Once the charts and dashboards are developed, they are routed to appropriate end users specific to the business function. The user then may choose to collaborate with peer business associates and take appropriate actions or responses. Some commonly used collaboration methods could be: the ability to send and receive comments, to raise formal issues, to route them and escalate them to appropriate levels and resolve them, to take appropriate action items, and to track the successful execution and completion of action items, define and track milestones to achieve metric targets, and in some cases even attach to a formal Root Cause Analysis process. Today largely, this process is in isolation from computer tools, and it is predominantly by manual. In a nutshell, it is important to note that all of the above five stages are meant to be part of a process to get raw data to the boardroom as a meaningful set of Key Performance Indicators (KPI) metrics. But today, the process to get raw data to the boardroom are executed in silos, using software developers having expertise in different development toolboxes (for example, ETL tool comes with a developer toolbox, RDBMS is a data repository toolbox for database developers and administrators, business intelligent tool is a charting toolbox for developers in developing graphs).
It is important to note that these five stages cannot be done by end users, since they are not trained of the development toolboxes nor do they have programming knowledge. As one might note, there is no single, comprehensive, end-to-end solution that exists, especially that which can be set up, monitored and used directly by end users, without programming intervention, either to setup or to operate. Instead, many enterprises implement custom solutions, or metrics initiatives, each specific to their organization, specific to their business need, by combining functionalities of these five layers of tools. Thus, invariably, such glued frameworks end up as a mixed bag of five layers of heterogeneous systems. Consequently, they place a heavy burden and continued dependence on information technology (IT) Subject Matter Experts (SMEs) for ongoing development, support and enhancement. In current information metrics systems, there is no general purpose, configurable, end-to-end web-enabled, automated solution that can serve not only as a metrics collection, storage and reporting tool, but serve as a two-way conduit for communication and collaboration. In essence, an integrated metrics-driven management control system does not exist.
The unavoidable need to use of five types of tools is not without reason. So far, the industry has not been able to visualize and combine these five stages into a cohesive unit that can be setup, operated and used by end users, without programming assistance, because: In every case, the underlying source data structures are different. In every organization, the different business functions have their own systems, each having its own relational data structures unique to serve its needs. Therefore, aggregating metrics from those systems also ends up as a custom aggregated data mart for metrics. There has not, thus far been a universal data warehouse which can take the place of relational data structures, as a generic, denormalized equivalent. Nor did emerge a “universal data warehouse” that is configurable by the user, so that the user can define a set of metrics, define rules and store data for the metrics, along with drillable data from multiple sources. Historically, the adherence to “Logical Data Model” principles, such as referential integrity, foreign keys, normalization etc., resulted in a huge set of “relational tables” that are interconnected via foreign keys, links, joins and relationships. In addition, such large set of tables would require complex SQL statements, unique to every query. Practices such as “cost optimization”, “space optimization” got into the realm of SQL, to save storage, but unable to produce faster results, since the SQL has to access many inter-related tables via joins and sub-queries anyway. So, decades ago, database designers came up with a practical design compromise called “denormalization” technique. This Denormalization exercise was done by database designers, AFTER they came up with the ideal “logical data model”. While it is still an accepted industry practice, every “denormalized model” was unique pertaining to the nature of the application. SO far, a generic denormalized model that can accept ANY logical data model as input has not emerged.
Even though ETL commands exist, they serve more as APIs or used by developers interactively through screens. For every data coming from every source, the ETL developer has to map via ETL tool, to a specific target in the data warehouse, custom designed for the organization. So far, there are no ETL command sets that the user can give on his/her own, mapping to a generic, universal data structure. Even though business intelligent (BI) tools can create “BI charts” that can be dragged and dropped into dashboards by developers, still the data grids and corresponding “drill downs” and other intelligent “analytics possibilities” have to be pre-coded and put in a platter by the developers as “analytics visualizers” and handed over to end users. Once they get such “analytics visualizers”, end users can operate them in seemingly quasi “self-service” mode, but ONLY within the confines of slice, dice and narrow possibilities offered by the developers. To use object-oriented analogy, even though charts exist, there are no “BI plug-in” intelligent reusable objects in existence, that are not only charts, but also contain data grids, and drill down information and metrics alerts. Therefore, the user cannot come up with a new set of metrics, design his/her own visualizers, that can be put together like building blocks to construct dashboards that can automatically offer drill down to the raw data levels.
End users have not been able to create and organize metrics and tie them to the raw data via computations, and configure them as “BI plug-in” objects. They have not been able to attach further “business collaborative intelligence” to the charts, as reusable objects, so that the objects carry the pointers to comment threads, issues, action items, milestones and root cause analysis processes. Neither have they been able to create and conduct meetings online, where the meeting agenda comprises the status of the collaboration mentioned above. Nor have they been able to automatically create meeting minutes—and store them for later use, such as for Legal compliance.
An end-to-end “metrics governance process” need to consist of a single cohesive platform with at least two components: a) to collect, compute, report on metrics and to collaborate and b) to take business decisions as a “closed business control loop”.
To design such a management control system involves addressing these possibilities: What if there exists a “universal denormalized database structure” that can readily accept sets of source data files into one simplified target relational data model? What if there exists an “end-user friendly” set of ETL commands and an ETL parser that can load a plurality of source data, into the above “universal data warehouse”? Assuming such a “universal data warehouse” can be populated directly by the user by the ETL commands, what if there exists a set of “BI plug-in” reusable objects that have “true business intelligence” built-in, (charts, data grid, drill down, metric alerts, collaboration—all in one) What if such a BI object can offer intuitive automatic drill down capabilities in many ways, depending on the raw data and the pertinent dimensions? On the other side, what if there exists a relational data model to define and organize metrics, according to business relevance, and with rules for allowed band limits and red/Yellow/Green conditions? What if every such metric can be tied to the raw data and/or derived data or aggregate expression from the universal data? And what if one can pass that metric as a parameter to the BI plug-in object mentioned in (3) above? 7. Bringing all of the above together, what if there exists a mechanism to establish many pointers or link-ids from each one instance of the “BI plug-in” object, link the object to comment threads, or to issue resolutions, or to an action item tracking system, or even to Root Cause Analysis system. The above questions resulted in formulating this invention.